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Despite  our  successes  over  the  last  decade  in  fielding 
dramatic  increases  in  joint  intelligence,  surveillance, 
and  reconnaissance  capabilities  and  compressing  the 
find-fix-target-engage-assess  timeline,  we  continue  to 
have  many  challenges  in  the  joint  interoperability  re¬ 
gime.  That  was  the  reason  the  2004  Testing  in  a  Joint  Environment 
Roadmap  recommended  establishing  a  DoD-wide  distributed  test 
infrastructure.  That  recommendation  led  to  the  establishment  and 
subsequent  rapid  growth  of  the  Joint  Mission  Environment  Test 
Capability  (JMETC),  a  distributed  test  program  launched  in  fiscal 
year  2007  that  is  designed  and  funded  to  support  Department  of 
Defense  programs.  However,  many  program  managers  and  sys- 
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Items  engineers  in  the  DoD  acquisition  community  may  be  unaware 
of  JMETC,  and  that  lack  of  awareness  results  in  a  lost  opportunity 
because  well-planned  and  well-executed  distributed  testing  can 
significantly  reduce  program  risk  and  increase  operational  effec¬ 
tiveness.  This  article  will  provide  program  managers  and  systems 
engineers  with  an  introduction  to  the  advantages  of  distributed  test 
and  the  benefits  JMETC  can  provide  to  their  program. 

Why  Distributed  Test? 

Program  managers  should  consider  conducting  a  distributed  test 
based  on  three  principal  advantages: 

•The  ability  to  reduce  overall  programmatic  interoperability  risk. 
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JMETC  is  the  DoD  corporate 
program  that  provides  the 
necessary  test  infrastructure 
to  conduct  joint  distributed 
events  by  cost-effectively 
integrating  live,  virtual,  and 
constructive  test  resources 
to  support  a  program's  needs 
for  assessments  and  tests. 


■  The  ability  to  identify  deficiencies  early  on,  and  finding 
and  fixing  problems  early  in  the  program  life  cycle  will 
have  much  less  impact  on  cost  and  schedule  than  defi¬ 
ciencies  identified  in  initial  operational  test  and  evalua¬ 
tion  (IOT&E). 

■  The  ability  to  efficiently  assess  and  test  the  system  in  its 
joint  context  early  on,  with  the  potential  for  early  assess¬ 
ment  from  operational  testers. 

Testing  is  an  expensive  endeavor;  and  testing  in  a  systems- 
of-systems  environment  is  inherently  more  expensive 
because  of  the  requirement  to  bring  multiple  systems  to¬ 
gether  to  verify  data  link  interoperability  and  create  a  real¬ 
istic  environment  that  provides  friendly  command,  control, 
communications,  computers,  intelligence,  surveillance,  and 
reconnaissance  (C4ISR)  systems  as  well  as  threat  capabili¬ 
ties.  Many  of  our  major  weapons  systems  and  systems  in 
development  have  high-fidelity  hardware-in-the-loop  sys¬ 
tems  that  use  actual  integrated  hardware  and  software  in  a 
lab  environment  and  accurately  replicate  weapons  system 
performance,  simulating  a  weapons  system  into  behaving 
as  though  it  is  receiving  real-world  inputs  and  outputs.  In¬ 
tegration  of  hardware-in-the-loop  systems  throughout  the 
continental  United  States  and  overseas  across  a  wide  area 
network  in  realistic  mission  environments  offsets  the  signifi¬ 
cant  cost  and  coordination  burden  associated  with  bringing 
the  systems  together  physically. 

When  conducting  interoperability  testing,  an  unmanned  aer¬ 
ial  system  may  need  to  work  with  Marine  and  Army  tactical 
units  on  the  ground  as  well  as  other  airborne  systems  such 
as  the  Air  Force's  Airborne  Warning  and  Control  Systems, 


the  Navy's  E-2  Hawkeye,  and  the  Joint  Surveillance  and  Tar¬ 
geting  System.  They  may  also  need  to  be  tested  with  other 
weapons  systems  with  which  they  will  interact,  such  as  the 
Marine's  F-18,  the  Air  Force's  F-16,  or  the  Army's  Advanced 
Field  Artillery  Tactical  Data  System  Joint  Fires  system.  All  of 
those  C4ISR  and  weapons  systems  have  test-quality  hard- 
ware-in-the-loop  simulators  with  current  software  available 
for  testing  through  distributed  means. 

Not  only  is  it  smart  to  assess  interoperability  early  in  the  de¬ 
velopmental  cycle  to  reduce  program  risk,  it  is  also  required 
by  DoD  Instruction  5000.02,  which  states,  "During  DT&E 
[< developmental  test  and  evaluation ],  the  materiel  developer 
shall  assess  technical  progress  and  maturity  against  criti¬ 
cal  technical  parameters,  to  include  interoperability,  docu¬ 
mented  in  the  TEMP  [ test  and  evaluation  master  plan].”  Ad¬ 
ditionally,  DoD  Instruction  5000.02  states,  "All  DoD  Major 
Defense  Acquisition  programs,  programs  on  the  OSD  T&E 
Oversight  list,  post-acquisition  (legacy)  systems,  and  all 
programs  and  systems  that  must  interoperate  are  subject 
to  interoperability  evaluations  throughout  their  life  cycles  to 
validate  their  ability  to  support  mission  accomplishment." 
That  policy  indicates  DoD  senior  leadership  is  serious  about 
joint  interoperability. 

Looking  beyond  basic  interoperability,  once  a  system's  hard- 
ware-in-the-loop  capabilities  are  integrated  for  distributed 
test,  those  same  systems  can  be  linked  together  to  address 
specific  mission  threads,  such  as  intelligence,  surveillance, 
and  reconnaissance  support  to  troops  in  contact  in  an  urban 
setting,  time-sensitive  targeting  using  simulated  weapons, 
or  ground  convoy  overhead  escort.  Again,  DoD  Instruction 
5000.02  provides  common  sense  guidance:  "Systems  that 
provide  capabilities  for  joint  missions  shall  be  tested  in  the 
expected  joint  operational  environment."  Distributed  test¬ 
ing  can  allow  operational  testers  to  execute  early  assess¬ 
ments  of  those  mission  threads  during  the  developmental 
test  phase,  providing  feedback  to  the  program  for  suggested 
changes  that  may  be  implemented  prior  to  IOT&E.  Alter¬ 
natively,  where  systems  are  performing  well  early  on,  the 
operational  test  community,  armed  with  previous  early 
exposure,  enters  IOT&E  with  a  higher  level  of  confidence 
in  the  program's  capabilities  and  limitations  and  can  tailor 
IOT&E  appropriately,  in  some  cases  potentially  saving  pro¬ 
gram  dollars. 

As  programs  transition  to  IOT&E,  the  focus  should  shift  to 
live  operations;  and  distributed  testing's  role  at  this  point 
may  shift  to  augmentation,  which  can  help  overcome  im¬ 
pediments  during  live  operations.  For  example,  in  testing 
future  unmanned  aircraft  systems,  many  will  have  a  require¬ 
ment  to  integrate  with  JSTARS,  which  tracks  surface  targets 
over  a  wide  area  and  can  provide  the  unmanned  aircraft  sys¬ 
tems  operator  with  increased  situational  awareness.  JSTARS 
deployment,  however,  is  costly  in  terms  of  identifying  un¬ 
manned  aircraft  systems  test  sites,  and  limited  state-side 
availability  will  preclude  live  JSTARS  test  support  in  many 
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cases.  Alternatively,  the  JSTARS  high-fidelity  hardware-in- 
the-loop  capability  is  persistently  on  the  JMETC  network, 
providing  higher  availability  and  much  lower  cost  than  live 
JSTARS  in  support  of  joint  and  unmanned  aircraft  system 
testing.  There  are  several  impediments  to  meeting  the 
DoD  Instruction  5000.02  mandate  that  systems  be  tested 
in  their  expected  joint  operational  environment.  The  first  is 
that  many  systems  operating  in  such  an  environment  are 
low-density/high-demand  assets  that  may  not  be  available 
at  all  or  for  the  required  amount  of  time  for  realistic  testing. 
The  second  issue  is  that  even  if  all  assets  are  available,  the 
cost  associated  with  deploying  multiple  assets,  maintenance 
support,  and  spares  can  be  significant.  Therefore,  augment¬ 
ing  the  system  under  test  with  high-fidelity  virtual  and  con¬ 
structive  systems  can  enable  one  to  create  the  realistic  joint 
environment  needed  to  properly  test  the  system.  Moreover, 
prior  to  live  testing,  virtual  and  constructive  systems  can  be 
used  to  rehearse  and  refine  the  live  test  plan. 

Improving  Test  Infrastructure 

JMETC  is  the  DoD  corporate  program  that  provides  the  nec¬ 
essary  test  infrastructure  to  conduct  joint  distributed  events 
by  cost-effectively  integrating  live,  virtual,  and  constructive 
test  resources  to  support  a  program's  needs  for  assessments 
and  tests.  JMETC  consists  of  a  core  reconfigurable  infra¬ 
structure  with  associated  products  and  customer  support 
that  enables  the  rapid  integration  of  live,  virtual,  and  con¬ 
structive  resources  to  link  systems  and  facilities  needed  for  a 
joint  testing  environment.  JMETC  is  currently  integrated  with 
40  test  and  hardware-in-the-loop  sites,  with  planned  expan¬ 
sion  to  approximately  60  sites  over  the  2010-11  timeframe. 
The  network  is  optimized  for  test  with  very  high  through¬ 
put,  low  latency,  and  negligible  data  loss.  The  network  has 
a  common  networking  protocol  and  middleware  optimized 
for  test  that  is  compatible  through  gateways  with  legacy 
simulations  and  facilities.  It  also  has  an  associated  collection 
of  high-performance,  primarily  government  off-the-shelf 
software  applications— known  as  JMETC  Tools— that  help 
JMETC  improve  test  planning,  management,  and  analysis 
capabilities  while  ensuring  required  network  performance 
is  maintained.  JMETC  Tools  also  include  the  command, 
control,  and  communications  assessment  tools  that  aid  in 
assessing  interoperability— the  same  tools  used  by  the  joint 
community  to  assess  interoperability  for  certification.  The 
JMETC  Web  portal  provides  information  on  distributed  test 
procedures,  upcoming  test  events,  tool  and  software  access, 
site  status,  lessons  learned,  and  help  desk  contacts.  Finally, 
JMETC  provides  an  expert  team  that  will  assist  in  planning 
and  supporting  distributed  test  events.  That  expert  team 
brings  procedures,  methodologies,  and  solutions  that  have 
already  been  tested,  proven,  and  put  into  practice. 

The  principal  mechanism  for  direction  of  the  JMETC  pro¬ 
gram  is  the  quarterly  JMETC  users  group  meetings.  The 
JMETC  program  relies  heavily  on  the  collaboration  of  the 
Services,  U.S.  Joint  Forces  Command,  and  other  test  and 
evaluation  agencies  to  build  an  infrastructure  relevant  to 


current  and  future  requirements.  In  order  to  facilitate  and 
formalize  this  exchange  process,  the  JMETC  Program  Of¬ 
fice  instituted  the  JMETC  users  group.  The  group  is  com¬ 
posed  of  representatives  from  acquisition  program  offices, 
technical  experts,  labs,  test  facilities,  and  ranges  that  use 
or  will  potentially  use  JMETC  infrastructure  and  products. 
Its  focus  is  on  technical  requirements  and  solutions.  The 
users  group  makes  recommendations  to  resolve  JMETC 
technical  issues  and  improve  integration  capabilities,  to  in¬ 
clude  connectivity  and  modernization  issues,  middleware 
and  object  model  requirements,  and  change  coordination. 
The  users  group  meetings  are  scheduled  quarterly  and 
dates  are  posted  on  the  JMETC  Web  portal  at  <https:// 
www.jmetc.org>.  First-time  users  will  have  to  register  on 
the  portal,  with  approval  normally  taking  several  hours  at 
most.  Program  managers  should  have  appropriate  repre¬ 
sentatives  begin  attending  JMETC  users  group  meetings 
as  soon  as  they  see  the  potential  need  to  conduct  distrib¬ 
uted  test  and  evaluation  as  part  of  their  programs. 

Distributed  Test  Examples 

There  are  several  programs  and  test  and  technology  ini¬ 
tiatives  that  have  leveraged  distributed  testing  and  the 
JMETC  program.  One  of  the  best  examples  is  the  Joint 
Surface  Warfare  Joint  Capability  Technology  Demonstra¬ 
tion.  JSuW  focuses  on  leveraging  traditional  intelligence, 
surveillance,  and  reconnaissance  assets  to  provide  long- 
range  guidance  to  net-enabled  weapons  in  high-threat 
littoral  environments,  posing  new  challenges  and  require¬ 
ments  for  data-link  functionality  and  concepts  of  opera¬ 
tions.  During  JSuW's  February  2009  SIMEX  [. simulation 
exercise]  event,  JMETC  partnered  with  the  Defense  Infor¬ 
mation  Systems  Agency  to  connect  three  separate  sites 
for  one  week  of  focused  events  to  simulate  the  littoral 
warfighting  environment,  with  virtual  F-18s  in  the  Boeing 
Center  for  Integrated  Defense  Simulation  in  St.  Louis,  Mo., 
constructive  P-3s  in  the  MITRE  Naval  C4ISR  Experimenta¬ 
tion  Laboratory  in  McLean,  Va.,  and  the  Virtual  JSTARS 
at  Northrop  Grumman  in  Melbourne,  Fla.  The  exercise, 
which  included  hundreds  of  tactical  engagements,  enabled 
the  JSuW  team  to  validate  their  more  mature  data  link 
message  sets  associated  with  net-enabled  weapons  and 
evolve  their  concept  of  operations. 

According  to  Bobby  Cornelius,  the  U.S.  Navy  lead  and 
JSuW  JCTD  program  manager,  “Because  of  the  dedication 
and  expertise  of  the  JMETC  team,  the  simulated  exercise 
stayed  up  and  was  stable  all  week,  allowing  us  to  execute 
all  desired  scenarios."  The  JSuW  team  will  continue  to  use 
distributed  testing  to  assess  the  full  suite  of  net-enabled 
weapons-related  data-link  messages  that  provide  control 
and  guidance  commands  until  the  JSuW  effort  transitions 
to  live-fly  in  this  fiscal  year. 

Another  example  is  the  U.S.  Air  Force  Global  Cyberspace 
Integration  Center,  which  conducts  Joint  Expeditionary 
Force  Experiments  (JEFXs)  for  concept  development,  ad- 
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vanced  technology  initiatives,  and  early  acquisition  test¬ 
ing  of  net-centric  capabilities.  JEFX  initiatives  include  net- 
enabled  weapons  and  network  interoperability  focusing 
on  airborne  networking  integration.  The  experimentation 
program  enables  early  informal  operational  assessments 
by  the  test  agencies  that  will  use  the  same  processes,  pro¬ 
cedures,  and  tools  used  in  JEFX  later  in  program  of  record 
formal  testing.  JEFX  has  a  10-year  history  of  aggressively 
using  distributed  live,  virtual,  and  constructive  opera¬ 
tions  and,  in  fiscal  year  2009,  determined  that  JMETC 
was  the  optimal  path  to  provide  the  required  tools,  con¬ 
nectivity,  and  on-demand 
network  infrastructure  for 
JEFX's  continuous  experi¬ 
mentation  requirements. 

By  leveraging  JMETC,  the 
Global  Cyberspace  Inte¬ 
gration  Center  has  saved 
an  estimated  $4  million 
in  fiscal  year  2009.  The 
savings  were  predomi¬ 
nantly  manpower  related, 
achieved  by  outsourcing 
expanding  connectivity 
requirements  to  JMETC 
and  by  transitioning  from 
the  extensive  coordination 
(and  manpower)  involved 
with  temporary  networks 
to  the  streamlined  coor¬ 
dination  associated  with  a 
persistent  network. 

The  Army  has  also  done 
extensive  distributed  in¬ 
frastructure  testing  using 
JMETC  to  prepare  for 
Future  Combat  Systems 
testing  and  the  follow-on 
Brigade  Combat  Team  modernization.  For  example,  the 
Army's  2008  Joint  Battlespace  Dynamic  Deconfliction 
Event  was  designed  to  investigate  and  verify  test  meth¬ 
odologies  to  asses  near-real-time  joint  airspace  command 
and  control  processes  during  Joint  Close  Air  Support  and 
Joint  Fires  operations,  including  assessment  of  airspace 
deconfliction.  Eighteen  separate  sites  were  integrated  for 
the  event.  Joint  Battlespace  Dynamic  Deconfliction  was 
supported  by  several  partner  Service  and  joint  initiatives 
from  the  U.S.  Army,  U.S.  Air  Force,  U.S.  Navy,  U.S.  Joint 
Forces  Command,  and  the  Office  of  Secretary  of  Defense; 
and  it  will  provide  a  framework  for  future  Army  and  joint 
modernization  testing. 


Maritime  Surveillance  unmanned  aerial  vehicle  and  the 
Air  Force's  Battlefield  Airborne  Communications  Network, 
a  U.S.  Central  Command  joint  urgent  operational  need 
program. 

Is  Distributed  Test  on  Your  Horizon? 

If  you  are  considering  distributed  testing,  or  are  already 
committed  to  distributed  testing  but  want  to  explore  op¬ 
tions  with  JMETC,  contact  the  JMETC  Program  Man¬ 
agement  Office.  Any  inquiries  can  be  made  by  sending 
an  e-mail  to  JMETC-feedback@jmetc.org,  and  you  will 

receive  a  response  within 
two  business  days.  Other 
points  of  contact  are  avail¬ 
able  from  the  JMETC  Web 
portal  under  the  “Ques¬ 
tions,  Comments,  and  Sug¬ 
gestions"  section. 

JMETC  team  members  will 
work  with  your  program 
office  and  integrated  test 
team  to  determine  options, 
requirements,  and  re¬ 
sources  needed  to  execute 
optimal  distributed  testing. 
For  more  significant  efforts, 
JMETC  members  are  well- 
positioned  to  become  one 
of  your  program's  team¬ 
mates,  participating  in  test 
working  groups  and  assist¬ 
ing  in  writing  the  test  and 
evaluation  strategy  and 
test  and  evaluation  mas¬ 
ter  plan.  The  costs  to  use 
JMETC  will  vary.  For  small 
test  events,  there  may  be 
no  cost.  Please  note  that 
JMETC  institutional  funding,  combined  with  the  ability  to 
leverage  existing  infrastructure  and  software  tools,  makes 
the  cost  of  teaming  with  JMETC  significantly  less  than 
establishing  a  program-specific  network.  Finally,  JMETC 
team  members  encourage  potential  customers  to  attend 
the  JMETC  users  group  to  share  requirements  and  col¬ 
laborate  with  other  distributed  test  users. 

For  more  information  on  JMETC,  please  go  to  the  JMETC 
Web  portal  at  <https://www.jmetc.org>.  The  portal  will 
provide  specific  dates  on  the  June/July  2010  JMETC  users 
group  meeting. 


Testing  is  an  expensive 
endeavor;  and  testing  in 
a  systems-of-systems 
environment  is  inherently 
more  expensive  because 
of  the  requirement  to 
bring  multiple  systems 
together  to  verify  data  link 
interoperability  and  create  a 
realistic  environment. 


Use  of  JMETC  is  steadily  growing.  JEFX  initiatives  in  2010 

will  include  B-2  Bomber  link-16  integration  testing  and  The  authors  welcome  comments  and  questions  and  can  be 
assessment  of  new  close  air  support  capabilities.  Other  contacted  at  chip.ferguson@osd.mil  and  vincent. difronzo. 
fiscal  year  2010  testing  includes  the  Navy's  Broad  Area  ctr@osd.mil 
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